1. Field of Art
The invention generally relates to an indexing system and more specifically to improving the efficiency of lookup and retrieval from a stored index.
2. Description of the Related Art
Electronic video libraries may contain thousands or millions of video files, making management of these libraries an extremely challenging task. The challenges become particularly significant in the case of online video sharing sites where many users can freely upload video content.
To facilitate searching of a large video database in order to identify and remove copies of videos, the reference videos may be indexed into a searchable reference index. When an input video is received, a lookup is performed on the index to identify reference videos (or portions of reference videos) that have characteristics in common with the input video. Based on the retrieved information, one or more reference videos (or portions of reference videos) can be matched to the input video. A problem occurs when an index lookup returns a very large list of results for a particular input query. In such a situation, the system may be unable to handle the large data flow due to constraints such as processing power, memory, or network bandwidth. A conventional solution to this problem is to blacklist common input queries. Using this technique, input queries that would return long lists of results are omitted and no results are returned. However, blacklisting causes a complete loss of information corresponding to the blacklisted input queries and thus decreases the accuracy of the matching process. What is needed is an efficient lookup and retrieval system that maintains the accuracy of the matching process.